The present invention relates to methods and apparatus for synthesizing three-dimensional picture signals and producing two-dimensional visual displays based on such signals.
In general, TV receivers, receiver monitors and CRT displays used in conjunction with household game playing apparatus, personal computers and graphic computing devices provide two-dimensional displays. The signals or data are produced so that two-dimensional characters and the like are suitably arrayed and moved on a planar background with or without changes in shape. However, in such display techniques the ability to change either the background or the character, as well as the movement thereof, are limited so that the ambience of the game or other display cannot be enhanced.
Methods have been adopted for formulating pseudo-three-dimensional images. In such techniques, plural pictures of a character to be displayed are stored, each picture depicting the character as viewed from a different direction. At a given time, one such picture is selected and displayed depending on the viewpoint adopted for the image being displayed. In another aspect of this technique, two-dimensional pictures are superposed along an image depth direction in order to display a pseudo-three-dimensional picture. In generating or formulating picture data, sometimes a texture mapping process is carried out whereby a surface texture or pattern, such as a ground pattern, is affixed to a selected face of a polyhedron forming an element of the image. A further technique employed involves converting picture color data by means of a so-called color pickup table in order to change the color of the displayed image.
FIG. 1 provides a block diagram of a proposed household game playing apparatus. In the apparatus of FIG. 1, a central processor (CPU) 91 comprising a micro-processor fetches operating information from an input device 94, such as an input pad or a joystick, via an interface 93 and a main bus 99. As the operating information is fetched by the CPU 91, three-dimensional picture data stored in a main memory 92 are transferred by a video processor 96 to a source video memory 95 for storage therein.
The CPU 91 also transmits data to the video processor 96 indicating the sequence in which pictures represented in memory 95 are to be displayed. The video processor 96 reads out picture data from the source video memory 95 in accordance with the data supplied by the CPU 91 in order to create a superposed picture display.
Simultaneously with the display of the picture, an audio processor 97 outputs voice data from an audio memory 98 which is based on voice information in the fetched operating information and which is coordinated with the picture that is being displayed. For example, if the picture being displayed depicts a car crashing, the audio processor 97 outputs an appropriate crashing sound.
FIG. 2 illustrates a sequence for producing and outputting a three-dimensional picture by the household game playing apparatus of FIG. 1, using two-dimensional picture data. The sequence depicted in FIG. 2 serves to produce a three-dimensional picture depicting a cylindrical object on checkerboard background pattern.
In FIG. 2, the source video memory 95 stores a background picture 200 in the form of a checkerboard pattern and a sequence of rectangular pictures 201 through 204 each representing a cross section of the cylindrical object to be overlaid on the checkerboard background pattern at varying depths. The portions of the rectangular pictures 201 through 204 other than the cross sections of the cylinder contain data representing transparency.
The picture data stored in the source video memory 95 is read-out in accordance with an address supplied from a read-out table 101 of the video processor 96. The CPU 91 provides read-out address control data to the table 101 over the main bus 99 for indicating which address is to be output from the table 101. In addition, a sync generator 100 produces read-out timing signals which it supplies to the read-out address table 101 and which are matched with synchronization signals for the picture to be displayed so that the read-out addresses are supplied by the table 101 in the appropriate order and at the appropriate times for displaying the desired picture.
The read-out picture data from the source video memory 95 are received by a superposition processor 103 which serves to superpose the picture data in the appropriate sequence in accordance with sequence data supplied from a priority table 102 under the control of signals supplied by the CPU 91 over the main bus 99. Each of the pictures 200 through 204 is assigned a sequential ranking beginning with a lowest rank for the background picture 200 and advancing to a highest ranking for the picture 204, so that the picture data as output by the superposition processor 103 represents a superposition of the pictures 200 through 204 in the appropriate order for displaying the cylindrical object on the checkerboard background.
The data output by the superposition processor 103 is supplied to a transparent color processor 104 wherein the data of each of the data pictures 201 through 204 representing transparency are processed so that underlying data may be displayed. Once the data has thus been processed by the transparent color processor 104, the data is output thereby as picture data representing a three-dimensional picture VD.sub.0 as shown in FIG. 2.
FIG. 3 is a block diagram of a proposed picture formulating apparatus which carries out a texture mapping function. In FIG. 3, a central processing unit (CPU) 301 is depicted comprising a microprocessor or the like coupled with a main bus 309. A main memory 302 for storing programs and data, a video memory 303 for storing texture source picture data and a video memory 304 for storing display output data as formulated by the apparatus of FIG. 3, are coupled to the main bus 309. The CPU 301 reads out the texture source picture data from the video memory 303 with such modifications as are necessary in order to transform the texture data for mapping onto a display area of the video memory 303. The transformed texture data are written in the display area of the video memory 304 and later read therefrom and converted by a D/A converter 305 into analog signals which are output for displaying a picture.
FIGS. 4A through 4C depict a sequence of processing operations carried out by the picture formulating apparatus of FIG. 3. As shown in FIG. 4A, in the video memory 303 several texture source pictures provide original data to be transformed as described above for texture mapping, such a source texture pictures 311a, 311b and 311c. The main memory 302 stores a program for controlling the CPU 301 for specifying the particular texture source picture to be used at a given time as well as a read-out position therefor to which the CPU 301 responds by reading out the appropriate picture data from the preset locations in the video memory 303 and carries out the above-mentioned modifications thereon as designated by the program to produce the modified picture 312a as illustrated in FIG. 4B. A modified picture 312a is then written in a display area 321 of the video memory 304. The write addresses for the display area 321 are also designated by the program stored in the main memory 302.
A sequence of such read out and modification operations as described above is carried out until the data of a complete picture 313 has been generated and stored in the display are 321 of the video memory 304. The picture data 313 is then read out from the video memory 304 in accordance with addresses determined as described above in synchronization with the video synchronization signals and converted to analog form by the D/A converter 305 to produce analog output picture signals.
FIG. 5 provides a block diagram of a picture formulating apparatus which carries out a picture data converting function whereby color data is output from a conversion table in the form of a color lookup table in response to image data. This provides the ability to change the color of the displayed image without re-writing image data. In the apparatus of FIG. 5, the CPU 301, main memory 302, video memories 303 and 304 coupled by means of the main bus 309, as well as a D/A converter 305, are similar to the corresponding devices of the FIG. 3 apparatus and are not, therefore, further described herein. In the arrangement of FIG. 5, a conversion table memory 306 is also provided storing a conversion table such as a lookup table for converting output display picture data read out from the video memory 304. The converted data are output from the conversion table memory 306 to the D/A converter 305 for conversion to analog form and are supplied thereby to a video or image output.
With reference also to FIGS. 6A through 6C, a sequence of operations carried out by the apparatus of FIG. 5 in processing and outputting a picture is illustrated therein. The. CPU 301 reads texture source data from the video memory 303 and modifies the same to store the modified data in appropriate locations of the video memory 304 to construct the picture 313 as depicted in FIG. 6A in accordance with the data conversion operations discussed above in connection with FIGS. 3 and 4. That is, the picture data 313 of FIG. 6A corresponds to the like-referenced picture as depicted in FIG. 4C. However the picture data 313 stored in the video memory 304 is provided in an intermediate or temporary form which cannot be output as is thereby to produce a picture display.
Rather, the picture data as store in the video memory 304 is read out from its display area 321, as represented by the illustration of FIG. 6B and used for addressing conversion table 314 in the conversion table memory 306 to output a color corresponding to each input address which, in turn, corresponds to a pixel of a picture to be output. The conversion table 314 serves to convert the address of virtual data supplied thereto from the video memory 304 into actual picture data, in accordance with a process represented schematically by FIG. 6C. The converted or actual picture data is output from the conversion table memory 306 to the D/A converter 305 for conversion thereby to analog picture data, as represented by FIG. 6D.
It will seen from the foregoing that the output picture data is not the same as that generated in the source video memory, but rather a three-dimensional picture is produced by changing read-out positions of the two-dimensional picture data stored in the source video memory and superposing the data of a number of pictures as read out. Consequently, the ability to express a three-dimensional picture generated in this manner is limited. For example, when processing a three-dimensional object picture for display by means of a household game playing apparatus on a screen, it may not be possible to correctly express the three-dimensional object picture in this way. It may occur that, since the point of view of the operator with respect to the object picture changes after the three-dimensional object picture has been processed, either the position of the object may not be changed correctly or the method of changing the three-dimensional picture may not be carried out in an appropriate manner. Likewise, the position of the viewing direction according to which the three-dimensional object is depicted is thus limited, for example, so that a back side of the object picture cannot be depicted even though the position or the direction of view of the three-dimensional object picture has been changed so that the same should be visible. It may also happen that the screen display to the operator occurs discontinuously even when three-dimensionally continuous movement is depicted.
Since multiple object pictures or picture data are stored in order to depict an object or other picture from various directions in order to represent a three-dimensional picture, the amount of data which must be stored for this purpose becomes voluminous. In addition, since the stored two-dimensional picture data are presented as three-dimensional picture data, substantial processing time is required for formulating each picture and game execution speed is consequently limited adversely.
In addition, since the two-dimensional picture data stored in the source video memory must be modified in order to produce a three-dimensional picture, the picture read-out control for the source video memory is complex and difficult to implement.
A further difficulty is posed, for example, in the case of the apparatus of FIG. 3 for which it is necessary to provide a video memory 303 which is dedicated to storage of the source texture data, which thus hinders efforts to achieve size reduction in the apparatus as well as to hold down production costs therefor. In the case of the apparatus of FIG. 5, it is necessary to provide a table memory 306 dedicated to storage of the conversion table as well as a CPU bus for accessing the table, which likewise hinders efforts to reduce the size of the apparatus as well as to minimize its production costs.